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Projective  Transformations  for  Interior  Point  Methods,  Part  II: 

Analysis  of  An  Algorithm  for  finding  the 

Weighted  Center  of  a  Polyhedral  System 

Abstract 


In  Part  II  of  this  study,  the  basic  theory  of  Part  I  is  applied  to  the 

problem  of  finding  the  w-center  of  a  polyhedral  system  X  .  We  present  a 
projective  transformation  algorithm,  analagous  but  more  general  than 

Karmarkar's  algorithm,  for  finding  the  w-center  of  X   .  The  algorithm 
exhibits  superlinear  convergence.    At  each  iteration,  the  algorithm  either 
improves  the  objective  function  (the  weighted  logarithmic  barrier  function) 
by  a  fixed  amount,  or  at  a  linear  rate  of  improvement.  This  linear  rate  of 
improvement  increases  to  unity,  and  so  the  algorithm  is  superlinearly 
convergent.    The  algorithm  also  updates  an  upper  bound  on  the  optimal 
objective  value  of  the  weighted  logarithmic  barrier  function  at  each  iteration. 
The  direction  chosen  at  each  iteration  is  shown  to  be  positively  proportional 
to  the  projected  Newton  direction.   This  has  two  consequences.   On  the 
theoretical  side,  this  broadens  a  result  of  Bayer  and  Lagarias  regarding  the 
connection  between  projective  transformation  methods  and  Newton's 
method.    In  terms  of  algorithms  it  means  that  our  algorithm  specializes  to 
Vaidya's  algorithm  if  it  is  used  with  a  line  search,  and  so  we  see  that  Vaidya's 
algorithm  is  superlinearly  convergent  as  well.    Finally,  we  show  how  to  use 
the  algorithm  to  construct  well-scaled  containing  and  contained  ellipsoids 
centered  at  near-optimal  solutions  to  the  w-center  problem.   After  a  fixed 
number  of  iterations,  the  current  iterate  of  the  algorithm  can  be  used  as  an 
approximate  w-center,  and  one  can  easily  construct  well-scaled  containing 
and  contained  ellipsoids  centered  at  the  current  iterate,  whose  scale  factor  is  of 
the  same  order  as  for  the  w-center  itself. 
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L  Introduction 

Part  II  of  this  study  uses  the  projective-centering  and  the  local  improvement 
methodologies  developed  in  Part  I  [  3  ]  ,  to  develop  an  algorithm  for  finding  the 

A 

w-center  of  a  polyhedral  system,  which  is  the  solution  x    to  the  problem 

m 
Pw  :    maximize  F(x)  =  V     w4  In  (bj  -  AjX) 

i=l 

subject  to       Ax  +  s  =  b  (1.1) 

s>0 

Mx  =  g 

m 
where    w  =  (wj, . . . ,  wm)  are  positive  weights  that  are  normalized  to  V    w4  =  1, 

i=l 

and  F(-  )     is  a  (weighted)  logarithmic  barrier  function.  Problem  Pw  is  a 
generalization  of  the  analytic  center  problem  defined  by  Sonnevend  [  11  ],  [  12  ]  ,  for 
nonuniform  positive  weights  on  all  constraints.     This  problem  has  had  numerous 
applications  in  mathematical  programming,  see  Renegar  [  10  ],  Gonzaga  [  5  ],  and 
Monteiro  and  Adler  [  8  ],  [  9  ],  among  others. 

If  there  exists  a  point    x  e  X  =  {x  e  Rn    |  Ax  <  b,  Mx  =  g)  for  which  Ax  <  b, 

A 

and  if  X    is  bounded  ,  then  Pw  will  have  a  unique  solution   x  ,  called  the  w-center 

A 

of   X  .  To  be  more  precise,  we  should  say  that    x    is  the  w-center  of  X  (A,  b,  M,  g), 
since  the  solution  to  Pw   is  dependent  on  the  particular  polyhedral  representation  of 

A 

X  .     However,  as  in  Part  I  of  this  study,  we  will  refer  to     x    as  the  w-center  of  X  , 
where  it  is  understood  that  X  represents  a  specific  intersection  of  half-space  and 
hyperplanes. 


For  the  case  when  all  weights  w4  are  identical,  there  are  two  other 
algorithms  known  to  this  author  that  have  been  developed  for  the  w-center 
problem.   Vaidya  [  14  ]    has  developed  an  algorithm  that  constructs  the 
Newton-direction  from  the  current  iterate  and  then  performs  an  inexact  line  search 
in  this  direction.   He  shows  that  at  each  iteration,  there  is  either  constant 
improvement  in  F(x)  or  a  linear  rate  of  improvement  in  F(x),  and  so  his  algorithm 
exhibits  linear  convergence.   Censor  and  Lent  [  2  ]   present  a  primal-dual  algorithm 

A 

for  finding  the  center,  that  is  convergent  to      x  ,  but  not  necessarily  in  any  strong 
sense. 

The  algorithm  developed  in  this  paper  is  based  on  the  use  of  projective 
transformation  methods  to  w-center  a  given  point,  as  in  Section  rV  of  Part  I.   At  each 
iteration  the  polyhedron  X  =  (x  e  Rn    |  Ax  <  b,  Mx  =  g]      is  projectively  transformed 

to  a  polyhedron  Z    so  that  the  current  point     x  is  the  w-center  of    Z  .    A  search 
direction      d    is  then  determined.  This  direction  is  used  first  to  test  if  Pw  is  bounded 

or  unbounded.   If  it  is  bounded,  it  will  then  produce  an  (updated)  upper  bound  on 

A 

the  value  of  F(x) .    A  steplength  a    is  then  computed. 

The  algorithm  can  be  run  with  a  steplength  a   determined  analytically  at  each 
iteration,  or  by  the  use  of  a  line  search.  The  new  point  xNEW   is  then  determined  by 

projectively  transforming  the  point      x  +  a  d    back  from  Z    to  X  .     At  each 
iteration,  there  is  either  a  constant  improvement  in    F(x)   ,  or  a  linear  rate  of 
improvement  in  F(x).    However,  because  the  linear  rate  of  improvement  approaches 
one  in  the  limit,  the  algorithm  exhibits  superlinear  convergence.    The  search 
direction  determined  at  each  iteration  of  the  algorithm  is  positively  proportional  to 
the  projected  Newton  direction.   If  the  steplength  is  chosen  at  each  iteration  by  a  line 
search,  the  algorithm  specializes  to  Vaidya's  algorithm  (with  equal  weights),  and  this 
shows  (obliquely)  that  Vaidya's  algorithm  exhibits  superlinear  convergence,  verifying 


a  conjecture  of  Vaidya    [  15  ]    that  his  algorithm  might  exhibit  stronger  convergence 
properties. 

A 

As  was  shown  in  Section  II  of  Part  I,  at  the  w-center   x  of   X  one  can  construct 
an  inner  ellipsoid  E1N    and  an  outer  ellipsoid  EOUT    with  property  that 

A  A 

EIN  c  X  c  EOUT   ,  x    is  the  center  of  EIN  and  EOUT  ,  and    (EOUT   -    x  )  = 

—      —  A  — 

((1  -   w)/  w)(EIN   -    x),  where     w  =  min  [wj]   .  This  ratio  is  (m  -  1)  when  all 

i 

A 

weights  are  identical.  Thus   x    is  in  a  sense  a  w-balanced  point  of  X   .  Although  the 

A 

algorithm  we  present  converges  to  the  w-center     x     of  X  ,  it  may  never  reach  the 
w-center.   However,  after  a  fixed  number  of  iterations  it  will  exhibit  a  point   x   "close 
enough"  to  the  w-center,  in  the  following  sense:   at  the  point    x  ,  one  can  easily 
construct  ellipsoids  FIN  and  FOUT  ,  with  the  property  that  FIN  c  X  c  FOUT    ,  and 
(FOUT  -  x  )  =  (1.75/w  +  5)  (FIN  -  x  )  .  When  all  weights  are  identical,  then  this  ratio 
is  (1.75m  +  5)  which  is  O(m).  In  general,  the  order  of  this  ratio  is  0(1/ w) ,  which  is 
the  same  as  for  EIN  and  EQut  • 

The  paper  is  organized  as  follows:   Section  II  presents  the  projective 
transformation  algorithm  for  solving  the  w-center  problem   Pw  .    In  Section  III,  we 

prove  that  the  optimality  tests  and  unboundedness  tests  in  the  algorithm  are  valid. 
In  Section  IV,  we  prove  that  the  objective  value  bounds  produced  by  the  algorithm  at 
each  iteration  are  valid,  and  prove  that  the  algorithm  is  linearly  convergent  in  F(x). 
In  Section  V,  we  show  that  the  algorithm   exhibits  superlinear  convergence.   In 
Section  VI  we  show  the  relationship  between  the  algorithm  and  Vaidya's  algorithm 
[  14  ]  ,  by  showing  that  the  direction    d  determined  at  each  step  is  positively 
proportional  to  a  projected  Newton  direction.   In  SectionVH,  we  show  that  after  a 
fixed  number  of  iterations,  one  can  easily  construct  ellipsoids    FIN  and  FOUT    about 


the  current  iterate     x  with  the  property  that  F1N  c  X  c  FOUT    ,  and     (FOUT  -   x  )  = 
(1.75/w  +  5)  (FIN  -  x  )  .     Section  VEQ  contains  closing  remarks. 

EL         A  Projective  Transformation  Algorithm  for  Finding  the  w-center  of  X  . 

The  notation  and  conventions  used  here  are  exactly  the  same  as  in  Part  I  of 
this  study  [  3  ]  .  We  assume  the  reader  is  familiar  with  these  notation  and 
conventions.   We  also  will  cite  many  of  the  results  presented  in  Part  I. 

Let  the  given  data  (A,  b,  M,  g)  define  the  polyhedral  system 
X  =  {x  e  Rn    |  Ax  <  b,  Mx  =  g]  .  Let    w  e  Rm    be  a  given  vector  of  weights  satisfying 

w  >  0  and  normalized  so  that  eTw  =  1,  where    e  =  (1, . .  . ,  1)T  .    Our  interest  lies  in 
solving  the  w-center  problem     Pw   given  in  (1.1).  We  make  the  following 

assumptions  regarding  the  data: 

(2.1a)  The  matrix  A  is     m  x  n    and  has  rank    n. 

(2.1) 
(2.1b)    The  matrix  M    is    k  x  n    and  has  rank  k. 

These  assumptions  are  for  convenience,  and  if  A    or   M   lacks  full  rank,  then 
one  can  either  eliminate  variables  or  constraints,  or  one  can  replace  certain  matrix 
inverse  operations  with  pseudoinverse  operations  in  the  analysis. 

Exactly  in  the  spirit  of  Karmarkar's  algorithm  and  consistent  with  the  local 
improvement  algorithm  of  Section  V   of   Part  I,  we  have  the  following  algorithm  for 
solving  Pw  .    Let  the  data  for  the  problem  be  [w,  A,  b,  M,  g,   x,  e  ]  .    Here  w  is  the 

vector  of  weights  satisfying    w>0    and    eTw  =  1,  (A,  b,  M,  g)  are  the  data  for  the 
polyhedral  system  X  ,   x  is  the  starting  point,  which  must  satisfy    s  =  b-A  x>0  and 
M  x  =  g  ,  and  e  >  0     is  the  optimality  tolerance. 


Let    w  =  min/w1J.    Because  we  will  use  the  quantity     w/(l  -   w)  extensively 
i 

throughout  the  description  of  the  algorithm  and  the  subsequent  analysis,  we  define 

the  constant  k  =     w/(l  -   w)   for  convenience.   Also,  in  the  algorithm,  F*  is  an  upper 

bound  on  the  optimal  objective  value  of  Pw  . 

Algorithm  for  the  W-Center  Problem 

StepO     Set     w=min{Wi}.    Set  k  =  (  w/(l  -   w)) .    Set  F*  =  +°°  . 

i 

Step  1      Set     s~=  b  -  A  x  ,  y  =  AT  S_1w  . 

Step  2     (Projective  Transformation  of  constraints)   A  =  A  -  syT 

Step  3    (Compute  direction  in  Z  space) 

Let    d  be  the  solution  to  the  problem  (P3) :     maximize    -  yTd 

d 

subject  to    dT  A  S^W  S'1  Ad  <  k 

Md  =  0 

If  this  problem  is  unbounded,  then  stop.   Pw  is  unbounded. 
If   yT  d  =  0,  stop,    x    solves    Pw  . 

Step  4   (Perform  Boundedness  Tests  and  Update  Upper  Bound). 
Set  y  =  (-yT  d)  /k 

If   y   £  l/k,stop.    Problem    Pw  is  unbounded,  and   d  is  a  ray  of  X. 
If  Y  <  1   then  set  F*  =  min   {F*,  F(  x)  +  y  +    Y2/(2(l-Y))} 
If  Y  <  .08567  then  set  F*  =  min   {f*,  F(  x)  +  .669kY2} 

Step  5  (Compute  Steplength) 

Set  cc=  1  -    ; 

Vl+2Y 


Set  zNEW  =    x  +  a  d. 


Step  6  (Transform  back  to  original  space  X  ) 
-  znew  ~   x 

XNEW   =     X   +     1  +    y^NEW"    *> 


Step  7  (Stopping  Criterion)     Set    x  <-  xNEW  .     If    F*  -  F(  x)    <  e ,  stop. 
Otherwise  go  to  Step  1. 

This  algorithm  has  the  following  straightforward  explanation.   Note  first  that 
Pw  is  an  instance  of  the  canonical  optimization  problem  (5.1)  of  Part  I,  namely 

m 
P      :    minimize   Fq  p(x)  =    ln(q-pTx)  -  £  wt  ln(bj  -  AjX  ) 
x,s  i=1 

subject  to:  Ax  +  s  =  b 

s  >  0  (2.2) 

Mx  =  g 
pTx  <  q 

where    q  =1  ,  and  p  =  (0, .  .  . ,  0)T  ,  and  hence  F(x)  =  -F       (x)    =    -Fj  0  (x)  .    Thus  we 
can  proceed  with  the  local  improvement  algorithm  presented  in  Section  V  of  Part  I. 
Let   x  e  int  X  be  given,  let     s  =  b  -  A  x  ,  and  let    y  =  AT  S_1w  .    Then  we  can 
projectively   transform  X  to 

Z  =  {z  e  Rn  |  (A  -  syT)z  <  b  -  TyT  x ,  Mz  =  g} 

X  —     X 

with  the  function    z  =  g(x)  =     x  +  ~ =-  ,  as  in  (3.2),  (3.3)  and  (3.4)  of  Part  I. 

1  -yl(x-  x ) 


Problem  P        =    P1  0    is  transformed  to 


[l_yTx]_[_yT]z   )  _   £     w.lnt. 


subject  to  (A  -  syT)z    +   t  =   b  -    syT  x  (2.3) 

t  >  0 
Uz  =    g 

-yTz  <  1  -  yT  x 

which  is  equivalent  to  (2.2)  (and,  of  course  (1.1))  according  to  Lemma  3.2  (ii)  of 

Parti.    Note  that  (2.3)  is  also  an  instance  of  problem  (5.1)  of  Part  I  with  A  replaced 

by   A  =  A  -  syT ,  p  replaced  -y ,  q  replaced  by  1  -  yT  x  ,  etc.    Because    x  is  the 

w-center  of     Z  ,  then  the  direction     d  given  by  the  solution  to  (5.3)  of  Part  I  (with 

A  and  p  replaced  by  A    and  -y)  has  the  property  that     x  +   d  maximizes    -yTz 

over 

ze    EIN  =    {zg  Rn|Mz  =  g,(z-  x)T  AT  S-!W  S"1  A(z-   x)<    w/(l  -   w)}  . 

Let  y  =  -yT  d  (1  -  w)/  w     as  in  (5.6)  of  Part  I  (note  that     q  -  pT  x  =  1).    This  is 
precisely  the  quantity    y    defined  in  Step  4  of  the  algorithm.  Let  a  =  1  -1/\1  +  2y  , 
as  in  Step  5  of  the  algorithm,  and  zNEW  =    x  +  a  d.    Then  by  Corollary  5.1  of  Part  I, 

G(zNEW  )  =  G(  x  +  a  d  )  <  G(  x  )  -   (  w/(l  -  w))  (l  +  y-  V^+2y  )  if 

a  =  1  - 1/\  1  +  2y  .  Projectively  transforming  back  to  X  space  using  the  inverse  of 

z  —   x 

g(-  ),  namely  x  =  h(z)  =    x  +  = =-   (see  (3.4)  of  Part  I)  ,  we  obtain 

1  +  yUz-  x  ) 

—  ^NEW  —    x 

xnew  =    x  +  — =-     as  in  Step  6  of  the  algorithm,  and 

1  +  y  (znew  ~  x  ' 

F(xNEW)  -  F(  x)  =  F1#  0  (  x)  -    F^  0  (xNEW)  >  (  w/(l  -   w  ))  (l  +  y  -  Vl+2y  )  ,  (2.4) 
from  Lemma  3.2  (ii)  of  Part  I. 


In  particular,  whenever  y  is  greater  than  or  equal  to  a  given  constant,  and  we 
will  use  y  ^  -08567,   then  we  have 

F(xNEW)  -  F(  x)  >  (  w/(l  -  w  ))  (.0033)  (2.5) 

Before  proceeding  with  the  analysis  and  verification  of  the  algorithm,  we  make 
the  following  remarks. 

Remark  2.1.    Use  of  a  line  search.    Steps  5  and  6  of  the  algorithm  can  be  replaced  by  a 
line  search.    Because  the  projective  transformation   g(-  )    (  (3.3)  of  Part  I)  preserves 
directions  relative  to    x,  the  line  search  can  be  performed  in  the  space  X  directly. 
Specifically,  one  needs  to  find  a  value  of  8  that  nearly  maximizes  F(  x  +  5  d  )  over 
8  >  0    and     x  +  8  d  feasible.    Because  F(x)  is  strictly  concave  over    x  e  X  ,  then 
there  will  be  at  most  one  maximizer  of  F(  x  +  8  d  )  over  the  feasible  range  of  8  .  One 

could  start  the  line  search  with    8  = zr-=  ,  where    a  =  l-l/"Vl+2v    which 

1  +  ay1  d 

corresponds  to  a  step  of  size  a  =  1  -  l/*\  1  +  2y    in  the  projectively  transformed  space 

Z  . 

Remark  2.2.    Efficient  Computation  of     d  in  Step  3.     As  in  the  linear  programming 
algorithm  of  Part  I,  one  can  compute     d   without  working  with  the  possible  very 
dense  matrices    A  =  A  -  syT   or    Q  =  A  S'^W  S"1  A  .   The  discussion  of  this 
procedure  is  deferred  to  Section  VI,  where  we  show  that     d  is  proportional  to  the 
projected  Newton  direction. 

IIL       Optimality  Test  and  Unboundedness  Test 

In  this  section,  we  show  that  the  optimality  test  of  Step  3  of  the  algorithm  is 
valid,  and  that  the  unboundedness  tests  of  Steps  3  and  4  are  valid. 


Proposition  3.1.   If  the  algorithm  terminates  in  Step  3  because  optimization  problem 
(P3)  is  unbounded,  then  Pw  is  unbounded. 


w 


Proof:  Suppose  that  at  Step  3,  that  (P3)  is  unbounded.  That  means  there  exists  a  ray 

r  e  Rn  such  that    yTr  =  -1  ,  Mr  =  0,  and    Ar  =  0.    But    Ar  =  0  implies  Ar  =    syTr  = 

m 
-  s  <  0.  Thus    r  is  a  ray  of  X  ,  and  F(  x  +  6r)  =  V    Wj  In  (  s,  (1  +  6) )  ->  +°°      as 

i=l 

0   -»oo.  ■ 

Proposition  3.2.  If  the  algorithm  terminates  at  Step  3  with  yT  d  =0,  then    x  solves 

IV 

Proof:    The  solution     d   will  satisfy  the  following  optimality  conditions: 

-y  =  2  pAT  S"1  W  S"1  A  d  -  t^M  ,  where      p  >  0  and      p(k  -   cFa1  S^W  S'1  A  d)  = 
0.    We  also  obtain    (3  cFa7  S'1  W  S"1  A  d  =  (1/2)  (-yT  d  +    t^M  d)  =  (1/2)  (-yT  d)  = 
0,  so  that     P  =  0  or  A  d  =  0  .     In  either  case,    y  =   j^M  ,  i.e.     wT  S_1A  =    t^M 
Thus  from  (2.1)  of  Part  I,    x    solves  Pw  .   ■ 

Proposition  3.3.  If  the  optimization  problem  (P3)  in  Step  3  has  a  solution  with  a 
nonzero  optimal  value,  then  that  solution  is  unique. 

Proof:  If  not,  then  let  r  =  d1  -  d2      where    d1  ,  d2    solve  the  optimization  problem. 
Then  it  is  straightforward  to  show  that  Ar  =  0  and  Mr  =  0.    Because     A  =  A  -  syT , 
then  Ar  =    syTr  ,  and  yTr  *  0,  for  otherwise  A  would  not  have  rank  n.  However, 
yTr  =  y^d1  -  yTd2  =  0    because    d1  and  d2  are  both  optimal  solutions,  which  is  a 
contradiction.    ■ 

Proposition  3.4.    If  Algorithm  3  stops  in  Step  4,  then  Pw  is  unbounded,  and     d  is  a 
ray  of  X  . 


10 


Proof:     From  Theorem  4.1  of  Part  I,      x  is  the  w-center  of  Z   as  defined  in  (3.2)  of 
Part  I,  and     x  +    d  lies  in  the  inner  ellipsoid  EIN  for  Z   as  given  by  Theorem  2.1  of 

Parti.    Thus     x+adeZ  for  all  a  e  [-1,1].    In  particular,  let  ct  =  l/(ky).    Then  if 
Algorithm  3  stops  in  Step  4,  a  <  1  and    a  >  0,  whereby  z  =   x  +  adeZ. 

Because  z  e  Z  ,  (A  -   syT)z  <  b  -  syT  x  ,  so  that 

Az  <     syTz  +    b  -   syT  x    . 

vT  d 
However,  we  also  have  y1z  =  y1x  +  ayid=y1x  +  *- —  =    y1  x  -1.    Thus 

ky 

Az  <    s  (yT  x  -  1)  +  b  -  syT  x    =b-s  =  Ax. 

Thus  A(z  -   x  )  <  0  .    But  z  -  x    is  a  positive  scalar  multiple  of    d  ,  so  that  A  d  <  0. 
Furthermore      d  *  0  ,  for  otherwise  the  algorithm  would  have  stopped  in  Step  3  (see 
Proposition  3.2).      Next,  observe  that  M  d  =  0,  so  that    d  is  a  ray  of  X  .  Because  A 
has  full  rank  ,  (see  (2.1) ),  Ad<0  and    A  d  *  0.     Thus  F(  x  +  0  d)  ->  +~  as    6  ->  <« 
and    Pw  is  unbounded.    ■ 


Section  IV.    Linear  Convergence  and  Improved  Optimal  Objective  Value  Bounds 

The  purpose  of  this  section  is  to  establish  the  following  three  results  regarding 
the  algorithm  for  the  w-center  problem: 


1 1 


Lemma  4.1.  (Optimal  Objective  Value  Bounds)    At  Step  4  of  the  algorithm  , 


A 


(i)         if    y<1,  then  Pw  has  a  unique  optimal  solution     x    ,  and 
F(  x  )  <  F(  xj  +  y  + 


2  (1  -  y) 

Y  <  .08567,  then  F(  x  )  <  F(  xj  +  .669  kY^ 


A  -> 


Note  that  Lemma  4.1    validates  the  upper  bounding  procedure  presented  in  Step  4  of 
the  algorithm. 

Lemma  4.2  (Local  Improvement).   At  Step  6  of  the  algorithm, 

(i)        if   y  >  .08567,  F(xNEW)    >  F(  xj  +  (.0033)  k  . 

(ii)       if  y   <  .08567,      F(xNEW)  -    F(  x)    >  .4612  kY2  . 

Lemma  4.3  (Linear  Convergence).  At  each  iteration,  at  least  one  of  the  following  is 
true: 

(i)  F(xNEW)    >  F(  x)  +  (.0033)  k  . 

A  A  _  A 

(ii)        F(  x  )  -  F(xNEW)  <  .32  (F(  x  )  -  F(  x) ),  where    x    is  the  w-center  of  X  . 


In  SectionV,  we  will  show  a  result  that  is  stronger  than  Lemma  4.3,  namely 
_  A 

that  as  the  iterates    x  converge  to    x  ,  that  the  constant  .32  in  Lemma  4.3  (ii)  will  go 

to  zero,  thus  establishing  superlinear  convergence. 


1  2 


Note  that  Lemma  4.3   is  an  immediate  consequence  of  Lemma  4.1   and  4.2. 
Lemma  4.3(i)  is  a  restatement  of  Lemma  4.2(i).   To  prove  Lemma  4.3(11),  note  that  if  y 
<  .08567,  then  from  Lemma  4.1  (ii)  and  4.2(H), 

F(xNEW)  -    F(  x  )  .4612 

KJF = >    -T7g-    >    -68, 

F(  x  )  -  F(  x)  -bby 

and  so 

F(  x  )  -  F(xNEW)  F(xNEW)  -    F(  x  ) 

A Z —    =     1  -    * ~ ^    1  -  .68  =  .32  .  ■ 

F(  x  )  -  F(  x  )  F(  x  )  -  F(  x) 

We  thus  need  to  prove  Lemmas  4.1  and  4.2.    We  start  by  asserting  some  elementary 
inequalities. 


Proposition  4.1.   (Inequalities) 

f(h  -  ln(l  +  h)h 


a)         In  (1  +  x)  <  x    - 


»2 


x2    whenever  -1  <  x  <  h. 


b)  In  (1  +  x)  <  x  -  .378  x2    whenever    -1  <  x  <  .5  . 

c)  1  +  y  -  V 1  +  2y    >   I  VI  +  8  -  Vl  +29  )/Q2\y2      whenever    0<y<9. 

d)  1  +  y  -^1  +2y    ^    -4612  y2      whenever    0<y^  .08567. 

Proof:    (a)  follows  from  the  fact  that   [h  -  ln(l  +  h)  ]  /  h2    is  decreasing  in  h  for 

h  >  -1.  (b)  follows  from  (a)  by  substituting  h  =  .5.    (c)  follows  from  the  fact  that 
\1  +  0  -  Vl  +26  )  /  62    is  decreasing  in  6    for  0  >  0.    (d)  follows  from  (c)  by 

substituting   9  =  .08567    .   ■ 

We  now  will  prove  Lemma  4.2,  followed  by  Lemma  4.1(i)  and  Lemma  4.1  (11). 
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Proof  of  Lemma  4.2.:    Statement  (i)  is  a  restatement  of  inequality  (2.5).   According  to 
(2.4),  F(xNEW)  -   F(  xj    >  k  (l  +  y  -  "^  +  2y  )  .  Let    6  =  .08567.  Then  according  to 

Proposition  4.1(d),      l+y~Vl+2y  >  .4612  y2      for    0  <  y  <  .08567  ,  which  proves 
statement  (ii).    ■ 

Proof  of  Lemma  4.1. (i):    Let    x  be  the  current  point,  and  let  y  =  AT  S_1w  .    For  any 
x  e  X  ,  let  z  =  g(x)  where  g(-  )  is  the  projective  transformation  given  in  (3.3)  of  Part  I, 
and  let  Z     be  given  in  (3.2)  of  Part  I.  Then  Pw  is  equivalent  to  the  problem  (2.3): 

m 
minimize  G(z)  =  In  (1  -  yT  x  +     yTz  )    -  V    wt  In  tj 

i=l 

subject  to  (A  -  syT)z    +  t  =  b  -   syT  x 

t  >  0 

Mz  =  g 

-yTz    <  1  -  yT  x 

and  by  the  remarks  following  (2.2)  and  (2.3),  and  lemma  3.2(h)  of  Part  I,  F(x)  = 
-F}  0(x)  =  -G(z) .    It  thus  suffices  to  show  that  if    y  <  1,  then  G(z)  > 

G(  x  )  -  y   -  y2  /  (2  (1  -  y))  .  By  construction  of  y,  from  Theorem  4.1  of  Part  I  we  know 

m 
that    x  is  the  w-center  of  Z  .    Thus  for  any  z  €  Z  ,    -  Y    w4  In  tj    > 

i=l 
m 
-  V    Wj  In    si  =    G(  x  ) ,  where    t    is  the  slack  corresponding  to  z  in  Z   . 
i=l 

We  now  must  show    ln(l  -  yT  x  +     yTz  )  >    -y-  y2  /  (2  (1  -  y))  .    To  see  this,  note 
that  because     x  +   d  maximizes    -yTz     over    z  €  E1N  ,  then     x  +   d/k  maximizes 
-yTz    over   ze  EOUT    ,  and  so  for  any   ze  Z  c  EOUT    ,-yTz  <  yT  x  -yT  d/k  = 
-yT  x  +  y .     Thus        ln(l    -  yT  x  +    yTz)      >  ln(l  -y)     >    -y-  y2  /  (2  (1  -  y)) ,  from 
Proposition  2.5  of  Part  I.   This  proves  statement  (i)  of  Lemma  4.1  .   ■ 
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The  proof  statement  (ii)  of  Lemma  4.1    is  very  involved,  and  follows  from  the 
following  sequence  of  lemmas: 

Lemma  4.4.    Let  h  >  0    be  a  given  parameter.    Let    x  be  the  w-center  of  X  ,  let 
_  _  A 

s  =  b  -  A  x,  and  suppose     x  e  X     satisfies 

(  x-  x)T  AT  S^W  S^AC   x-   x  )  =  p2  . 

Then 


m  ^  m 

V    Wj  In  (b,  -  A,  x  )     -     V    Wj  In   Sj    <  * 

i=l  i=l 


Proof:  First  we  observe  that 


h-ln(l  +  hL,  r- 
± i>2  if    (3  <hVk 

h-ln(l  +  h)       r-  ,- 
-2 h^kp  if    p>  hVk 


m  ^  m  m 

V    Wj  In  (b,  -  AjX  )     -     ^    Wj  In  (  Sj )  =      X    wi  ^n  ^  +  ri^   '  wnere 
i=l  i=l  i=l 

r  =  -S"1A(x-  x).  Then  note  that    wTr  =  -wTS_1A(x-  x)  =  nTM(x-  x)  =  0 

for  some  k  =  Rk  ,  from    (2. Id)  of  Part  I.    Also,  rTWr  =  p2  .     Then 

(n/k  /  p  )T  W(rVk  /  p)  =  k ,  and  by  Proposition  2.2  of  Part  I,  |  r4 1  <  p  /  Vk  ,  i  =  1, . . . ,  m. 

We  now  prove  the  two  cases  of  the  Lemma. 

Casel.       (p<hVk).    Then    |rj|<h.    From  Proposition  4.1(a),  ln(l  +  ^  )  < 

(h-ln(l+h))    2  ™         ,    M 

Tj  -  ~2 rf  ,  i  =  1, .  . .  m  .     Thus       2^    wi  ^n  ^  +  ri)    - 

i=l 

T         (h-lnO  +h))   Tt  r            (h-lnO+h))     - 
w'r n r'Wr  = r? P^  . 


h2  «    r»i    -   -  h2 
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Case  2.      (P  >  hp/k  ) .     Because  |  r;  |  <  P  y[k,  then  r{  Vk  h/p  <  h,  and  so  again  by 

r—                       r-               (h  -  ln(l  +  h))     ,       -  ,„n 
Proposition  4.1(a),  ln(l  +  r,  Vk  h/p  )  <    r,  Vk  h/p ^ ri    kh2/p2, 

™                          r-                   (h-ln(l+h»      , 
i  =  1, . . . ,  m.  Thus      ^    wi  In  (1  +  ryvk  h/p  )  < r^ kh^  ,  because 

i=l 

wTr    =  0     and  rTWr  =  p2  .  However  by  the  concavity  of  the  log  function, 
mm  m 

(Vk  h/p)  X    w.  In  (1  +  rt)    =  (Vkh/P)  X  w.  In  (1  +  q)  +  (1  -  Vk  h/p)  £    In  (1) 
i=l  i=l  i=l 

m  m 

<  £    WjlnaVkh/pXl+rj)  +    (1-Vkh/P))=     X    w4  In  (1  +  rjVkh/P) 

i=l  i=l 

(h-ln(l  +  h))  ,    -       ^ 

< -5 kh2  .     Thus 

h 

£          ,    r,                     (h-ln(l  +  h))      r-    0      _ 
^    Wj  In  (1  +t{)  <    -  — — -2 Vk  hp  .  ■ 

i=l 


Lemma  4.5.  Let    x  be  the  current  point  in  the  w-center  algorithm,  let 

Q  =  AT  S_1W  S"1  A  ,    where      A  =  A  -  syT  is  defined  in  Step  2,  and  y  is  defined  as 

A  A  A 

in  Step  4.    Suppose   x    is  the  optimal  solution  to  Pw  and    z=g(x),  where  g(- )  is  the 

a         _     _   A         _ 
projective  transformation  given  by  (3.3)  of  Part  I,  and  that  (z  -    x )  Q(z  -    x)  =  P2. 

If  Y  <  1,  and  h  >  0  is  a  given  parameter,  then 

'-  (  h  "  ^  +  h))  P2   +  pVky  +  P2ky2/(2(1-Y))    if   p  <  hVk 


F(x  )  -  F(  x  )    <<\ 


'  —  hVkp    +    pVky  +  p2ky2/(2(l-Y))  if    P  >  hVk 


h2 


A  _  _  A 

Proof:     According  to  Lemma  3.2(h)  of  Part  I,  F(x  )  -  F(  x  )  =  G(  x  )  -  G(z  ) ,  where 

m 
G(z)  =  ln(l-yTx  +    yTz)  -     j     w;  In  (bi  -  A;z) 

i=l 
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where     A  is  defined  in  Step  2  of  the  algorithm,  and    b  =  b  -   syT  x  .    Noting  that 

m 
G(  x )  =  -F(  x)  =  -     j£     Wj  In  Sj,  then 


i=l 


a       _  m  _  a  m 


G(  x)-  G(z)    =  -ln(l+yT(z-   x))  +    £     Wj  In  (bj  -  Atz)  -    £     v?.  In   s,    .(4.1) 

i=l  i=l 


However,  because     x  is  the  w-center  of  Z  (defined  in  (3.2)  of  Part  I),  then  by 
Lemma  4.4, 


m  ~    A  m  _ 

£     Wj  ln(bj  -Ajz)  -    X     wi  ln   si      s  ' 

i=l  i=l 


(   (h-ln(l  +h))    ,  p- 

-- ^ P     if    MhVk 


(h  -  ln(l  +  h))       j-n  r- 

^ h^P    if    (3  >  hVk 


T  a        _  p2ky2 

It  thus  remains  to  show  that  -ln(l  +  yUz-  x))<    p\ky   + 


2(1  -y) 


A  A  _  _ 

Let      d  =  (z  -    x  ).     The  vector    d  in  Step  3  of  the  algorithm  is  that  vector  that 
maximizes  -yT(  x  +  z)  over  all  z  e  EIN  ,  as  discussed  in  SectionV  of  Part  I,  where 
EIN  =  {ze  Rn|Mz  =  g,  (z-  x)T  Q(z-  x )  <  k)   .  Thus 

-yT(  x  +   d)    >    -yT(  x+  d  Vk  /p  ),andso     yTd   >  pyT  d/Vk  .    However, 

because  y=  -yT  d/k    then   yTd    >  -pyVk  .    Also,    because 

Z  c  Eom  ={z  e  Rn  |  Mz  =  g,  (z  -  x  )T  Q(z  -  x )  <  1/k)     from  Theorem  2.1  of  Part  I, 

we  must  have     P2<l/k  ,  and  so  pVk  <  1.     Thus      yp  Vk     <    y  <1  .    This  then 
implies 


p.. *>i 


ta  p-  j-  p-^k 

-ln(l  +  y1  d  )  <    -ln(l  -  Pyvk  )  <    +py\k    +      — ,  the  last  inequality  being  an 

2(1  -  y) 

instance  of  Proposition  2.5  of  Part  I,  with  e  =  -yp  Vk    and    a  =  y .    ■ 
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Lemma  4.6.   Under  the  hypothesis  of  Lemma  4.5,  if 

12 


Y<  1  - 


ln(l  +  h) 

2h 


-  1/2 


V 


ln(l  +hV 


ln(l  +  hh 


r       In  1  • 


(4.2) 


then    (3  <  hVk 


Proof:  Suppose   p  >  hyfk   .     Then  from  Lemma  4.5, 


F(x)-F(  x)    <  f(y,p)  ,  where 
(h-ln(l  +h)) 


f(Y,P)  =  - 


h2 


hVkp    +    pVkY   + 


2(1 -y) 


Note  f(Y,P)  increases  in  y  for    P>0    and    0<y<1-    Straightforward  calculation 
reveals  that  f(y,  p)  =  0     if 


2-ln(l+h)/h-    V(ln(l  +  h)/h)2    +  2pVk  (1  -  ln(l  +  h)/h) 

Y=      F (4-3) 

2-pVk 

Thus  if    y  is  less  man  me  above  quantity,  then  f ( y ,  p )  <  0  ,  contradicting  the 

A 

optimality  of  x  .    Thus  y  must  be  greater  than  or  equal  to  the  expression  in  (4.3). 
Next,  borrowing  the  observation  in  the  proof  of  Lemma  4.5  that    0  <  pVk  <  1,  then 
the  expression  in  (4.3)  is  greater  than  or  equal  to 

2  -  ln(l  +  h)/h  -  Vdnd +h)/h)2  +    2(1  -  ln(l  +  h)/h) 


,     ,            ,           lnd  +  h) 
which  equals   1  - — ~r 1/2 


V 


ln(l  +h)1 


~¥^] 


r-                          ln(l+h)         ,        /rin(l  +  h)"|2         f       lnd  + 
Thus,if    p>hVk,then    yil— -^ -1/2"\[ h J    +  I1  " h~ 


KT 
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Proof  of  Statement  (ii)  of  Lemma  4.1.  :    Let  us  set  h  =  .5.  Then  the  expression  on  the 

RHS  of  (4.2)  is  then  greater  than  .08567.    Thus  if  y  <  .08567,  then  from  Lemma  4.6  and 

Lemma  4.5,  (3  <  5^jk  ,  and 

F(x)-F(x)    <-.3781p2  +    pVky   +      P2k*r  /  (2(1  -  y)) .    Let  us  define 

f(p)  = -.3781  p2  +     Vkyp    +    P2ky2  /  (2(1  -  y))  .    The  function  f(P)  is  quadratic  in  p, 

and  because  k  <  1    and  y  <  .08567,  then    ky2  /  (2(1  -  y))  <  .3781  ,  so  that  f(p)  is 

—              \ky 
concave.  Thus  the  largest  value  of  f(p)  is  given  by     P  = 


.7562-' 


ky2       ' 


with  f(p)  <  f(  p)  = 


ky2 


ky2 


(l-y) 


2ky2  2-f 

1.5124-  —       1.5124- 


5-    <    .669ky2    for   y<. 08567. 


(l-y) 
This  completes  the  proof  of  Lemma  4.1 


d-y) 


V.        Superlinear  Convergence 

In  the  previous  section,  we  showed  linear  convergence  of  the  algorithm,  with 
a  linear  convergence  rate  of  .32,  by  choosing  the  value    h  =  .5  and  applying 
Lemmas  4.4,  4.5,  and  4.6.    In  this  section  we  show  that  as  we  choose  h  >  0  and 
arbitrarily  close  to  zero,   then  the  linear  convergence  rate  goes  to  zero  in  the  limit, 
thus  showing  that  the  algorithm  is  superlinearly  convergent. 

We  first  present  some  elementary  facts  about  three  particular  functions. 


Proposition  5.1. 

Let  f(h)  =  1  -i^S  -  1/2 


Let  j(0)  =  Ll  +  6  -  Vl+26  J  /62 


ln(l  +h)l 


3" 


£- 


ln(l  +  hV 


for    6  >  0 


for  h  >  0.    (5.1) 
(5.2) 
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,  ,       (h-ln(l  +h)) 
Let  p(h)  = ^2 for     h>0  (5-3) 

Then    lim    f(h)  =  0 ,      lim   j(0)  =  .5 ,  and      lim    p(h)  =  .5     .  ■ 

h->0  0->O  h->0 


We  then  prove  the  following  three  propositions,  after  which  the  proof  of  superlinear 
convergence  easily  follows. 

Proposition  5.2.    For  any  h  >  0,  at  Step  6  of  the  algorithm,  if  y  <  f(h) ,  then 

"1  +  f(h)  -  Vl  +  2f(W 
f(xnfw)  -  F(  X  )    > 


ky2 


(f(h))^ 

Proof:  F(xNEW)-F(  x)  >    k\l  +  y-  V1  +  2Y  /  from  (2.4).    Now  substituting 

f(h)  for  6    in  Proposition  4.1(c),  we  obtain  the  desired  result.   ■ 

Proposition  5.3.    Suppose  h  >  0  and  sufficiently  small  and  f(h)  and  p(h)  are  defined 
as  in  Proposition  5.1.  At  Step  6  of  the  algorithm,  if  y  <  f(h),  then 


a            _                    ky2 
F(x  )  -  F(  x  )    < /f/v    2     , 


2k(f(h))- 
4P(h)"(l-f(h)) 


A 

where  x    is  the  optimal  solution  to  P 


w 


Proof:    Because  f(h)  is  just  the  expression  of  the  RHS  of  (4.2),  we  have  by  Lemma  4.6 
and  Lemma  4.5  that  if  y  <  f(h),  then 

F(x)-F(  x)    <  -p(h)[32    +  (3VkY  +  (32kY2/(2(l-7)).  (5.4) 

If  h  is  sufficiently  small  p(h)  is  approximately  .5  from  Proposition  5.1   and 

ky2  /(2(1  -  y))  <  (f(h))2  /(2(1  -  y))  <  (f(h))2  /(2(1  -  f(h))  is  approximately  zero.  Thus 


the  RHS  of  (5.4)  is  quadratic  and  concave  in  p.    Its  maximal  value  occurs  at 


P  = 


Vk' 


2p(h)  - 


kT2 

(1-Y) 


and  the  maximum  value  of  the  RHS  in  (5.4)  is  therefore 

k^ 


4p(h)  - 


2k?2 
(1-7) 


However  y  <  f(h) ,  so  we  have 


F(x  )  -  F(  x  )  < 


kY2 


A      fM         2k(«h»2 
4P(h)  "  (l-f(h)) 


Proposition  5.4.  For  h  >  0  and  sufficiently  small,  if    x     is  the  optimal  solution  to 
Pw  ,  then 

i)  if  y  ^  f(h) ,  then  F(xNEW)  -  F(  x  )  >  (l  +  f(h)  -  >/l  +  2f(h)  )  k 


ii)        if  y  <  f(h),  then 


F(x)  -  F(xNEW) 
F(x)  -F(  x) 


1- 


1  +  f(h)  -  Vl  +  2f(h) 


(f(h))^ 


2k(f(h))2' 

4P(h)  "  (1  -  f(h)) 


Proof:    If  y>  f(h),  then  from  (2.4),  we  know  that  F(xNEW)  -  F(  x  )  >  \1  +  y-V1  +  2y  )k 
>  (l  +  f(h)  -  Vl  +  2f(h)  )  k    since  1  +  6  -  ^l  +29     is  an  increasing  function  of  9  >  0. 
Statement  (ii)  follows  directly  by  combining  Propositions  5.2  and  5.3.  We  have  that  if  y 
<  f(h)  ,  then 
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F(x)-  F(xNEW) 
F(x)    -F(  x) 


=    1- 


F(xNEW)  -  F(  x  ) 
F(x)  -  F(  x) 


(      ■ 


<  1- 


1  +  f(h)-  Vl  +  2f(h) 


(f(h))' 


\ 


ky 


\ 


1 


4p(h)   - 


2k(f(h)) 
(l-f(h)) 


Cancelling  out  ky2   and  rearranging  yields  the  desired  result. 


Lemma  5.1.    The  algorithm  for  solving  Pw   exhibits  superlinear  convergence  in  F(x). 

Proof:       It  suffices  to  show  that  as  h  -»  0,  then  the  RHS  of  (5.5)  goes  to  zero.  As 
h  -»  0  ,  f(h)  -»  0   from  Propostion  5.1.   Then  note  that 

i  +  e-Vi  +  29 

with  6  =  f(h),  9  ->  0    as  h  ->  0    and   5 =    j(9)  ->  .5     as    9  -»  0 ,  by 

9 

Proposition  5.1.  The  last  term  of  (5.5)  is  4p(h)  -  2k(f(h))2/(l  -  f(h))  .    As    h  -»  0 

p(h)  ->  .5  and  f(h)  — >  0  by  Proposition  5.1.  Thus  the  entire  expression  (5.5) 

approaches    1  -  (.5)  (4(.5)  -  0)  =  0  .   ■ 


VI.       Analysis  and  Computation  of  the  Improving  Direction 

In  this  section,  we  show  that  the  direction    d  of  Step  3  of  the  algorithm  is  a 
positively  scaled  projected  Newton  direction.   As  a  byproduct  of  this  result,  the 
computation  of    d    can  be  carried  out  without  solving  equations  involving  the 
matrix    Q  =  AT  S^W  S"1  A,  which  will  typically  be  extremely  dense.   Vaidya's 
algorithm  for  the  center  problem  [  14  ]   corresponds  to  computing  the  Newton 
direction  and  performing  an  inexact  line  search.    Thus,  our  algorithm  specializes  to 
Vaidya's  algorithm  when  our  algorithm  is  implemented  with  a  line  search. 


?  ") 


Furthermore,  this  establishes  that  Vaidya's  algorithm  then  will  exhibit  superlinear 
convergence. 

Let    x  be  the  current  iterate  of  the  algorithm,  let      s  =  b  -  A  x,  and  y  =  AT  S'Hv 
and     A  =  A  -   syT  as  in  Steps  1  and  2  of  the  algorithm,  and  let  Q  =  AT  S"TW  S"TA  , 
and  Q  =  AT  S_1W  S'^A  .    From  (2.1a),  A  has  full  rank,  so  that  Q  is  nonsingular  and 
positive  definite.   Let  F(x)  be  the  weighted  logarithmic  barrier  function  of  Pw  given  in 

(1.1).  Then  the  gradient  of  F(-  )  at    x  is  given  by  -y  ,  i.e.,  VF(  x  )  =  -y,  and  the  Hessian 
of  F(-  )  at    x  is  given  by  -Q,    i.e.,    V2  F(  x  )  =  -Q. 

Thus  the  projected  Newton  direction     dN    is  the  optimal  solution  to 

maximize  -yTd  -  (1/2)  dTQd 
subject  to    Md  =  0 

and  the  Newton  direction    dN     together  with  Lagrange  multipliers  JtN  is  the  unique 
solution  to 

QdN    -  Mtkn  =  -y  (6.1) 

MdN  =  0 

Because  Q  has  rank  n  and  M    has  rank  k,  we  can  write  the  solution  to  (6.1)  as 

dN  =  -Q_1y  +  Q_1mtkn 

(6.2) 
where  jcn  =   (MQ^M^MQ^y 

It  is  our  aim  to  show  the  following 
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Lemma  6.1.  Let  dN   be  the  Newton  direction  given  by  the  solution  (6.1)  or  (6.2). 
Then  1  +  yTdN  >  0 , and 

(i)        if    1  +  yTdN  >  0,  and    dN  *  0 ,     d  =  dNVk  /  (^/dNQdN  )  is  the 
direction  of  Step  3  of  the  algorithm. 

(ii)       If    1  +  yTdN  >  0,  and    dN  =  0  ,  then    d  =  dN  =  0  is  the  direction  of 
Step  3  of  the  algorithm,  and  the  current  iterate     x  solves  Pw  . 

(iii)      if     1  +  yTdN   =  0,  then  the  optimization  problem  (P3)  of  Step  3  is 
unbounded,  and  Pw  is  unbounded. 

Remark  6.1.  Simplified  Computation  of    d.    Lemma  6.1  (i)  shows  that      d    is  just  a 
positive  scale  of  the  Newton  direction   dN   .  Thus  in  order  to  solve  for   d,  one  need 

not  solve  a  system  involving  the  possibly-very-dense  matrix  Q  .  Rather  one  need 
only  solve  the  equations  (6.1)  for    dN    and  then  compute   d  =  dNVk  /  y  dNQdN    . 

Remark  6.2.    Relation  of  Algorithm  to  Vaidya's  algorithm.    Lemma  6.1(i)  shows 
that    d   is  just  a  positive  scale  of  the  Newton  direction  dN   .   Suppose  the  algorithm 

is  implemented  with  a  line  search  replacing  Steps  5  and  6  ,  as  suggested  in  Remark 
2.1.  Then  because  the  projective  transformations  g(x)  and  h(z)  given  by  (3.3)  and 
(3.4)  of  Part  I  preserve  directions  from    x,  the  algorithm  direction  in  the  space  X 
will  be  dN  .    Therefore,  when  using  a  line  search,  the  algorithm  is  just  searching  in 

the  Newton  direction.   This  is  precisely  Vaidya's  algorithm   [  14  ],  when  all  weights 
Wj  are  identical.   And  because  the  complexity  analysis  of  Sections  rv  and  V  carries 

through  with  or  without  a  line  search,  we  see  that  Vaidya's  algorithm  exhibits 
superlinear  convergence. 


2  4 


Remark  6.3.  An  Extension  of  a  Theorem  of  Bayer  and  Lagarias.    In  [  1  ] ,  Bayer  and 
Lagarias  have  shown  the  following  structural  equivalence  between  Karmarkar's 
algorithm  for  linear  programming  and  Newton's  method:    First  one  can 
projectively  transform  the  problem  of  minimizing  Karmarkar's  potential  function 
over  a  polyhedron  X    to  finding  the  (unbounded)  center  of  an  unbounded 
polyhedron  Z  ,  where  Z    is  the  image  of  X    under  a  projective  transformation  that 
sends  the  set  of  optimal  solutions  to  the  linear  program  to  the  hyperplane  at 
infinity.     Then  the  image  of  Karmarkar's  algorithm  (with  a  line  search)  in  the  space 
Z   corresponds  to  performing  a  line  search  in  the  Newton  direction  in  the 
transformed  space  Z  .  Lemma  6.1  is  in  fact  a  generalization  of  this  result.  It  states 
that  if  one  is  trying  to  find  the  center  of  any  polyhedron  X   (bounded  or  not),  then 
the  direction  generated  at  any  iteration  of  the  projective  transformation  method 
(i.e.,  the  algorithm  of  Section  II)  is  a  positive  scale  of  the  Newton  direction.   Thus,  if 
one  determines  steplengths  by  a  line  search  of  the  objective  function,  then  the 
projective  tranformation  method  corresponds  to  Newton's  method  with  a  line 
search. 

Another  important  relationship  between  directions  generated  by  projective 
transformation  methods  and  Newton's  method  can  be  found  in  Gill  et  al.  [  4  ]. 

We  now  prove  Lemma  6.1   by  a  sequence  of  three  propositions. 
Proposition  6.1.    If    (dN  ,  rcN  )  solve  (6.1),  then  1  +  yTdN  >  0. 

Proof:  Note  that  from  (6.2),  we  have 

1  +  yTdN  =  i  _  yTQ^y  +  yTQ-1MT(MQ-1MTr1MQ-1  y  >  1  -  yTQ_1y  . 

We  thus  must  show  that     yTQ_1y  <  1.    Note  that  Q  =  AT  S_1W  S"1  A    is  positive 
semi-definite,  and  that     Q  =  Q  -  yyT .  Thus  0  <  yTQ_1QQ-1y  =   yTQ~1(Q-yyT)Cr1y  = 
yTQ-1y(l  -  yTQ-1y)-    Therefore  yTQ-1y  <  1,  completing  the  proof.  ■ 
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Proposition  6.2.    If     (dN  ,  7tN  )  solve  (6.1)  and    1+  yTdN  =  0,  then  the  optimization 
problem  (P3)  of  Step  3  is  unbounded,  and  Pw  has  no  solution. 

Proof:  From  the  proof  of  Proposition  6.1,  we  see  that  if  1  +  yTdN  =  0,  then 

1  -  yTQ-V  =  0  (6.3) 

and      y^^M^MQ^M^MQ-V  =  0.  (6.4) 

Let    r  =  -Q^y  .    From  (6.4)  we  have    ^M^MQ^M1)'^:  =  0,  whereby  Mr  =  0. 
Also     rTQr  =    rT(Q-yyT)r=    yTQ_1y  (1  -yTQ_1y)  =  °     from    (6.3).  Finally,  from 
(6.3)  we  have  -yTr  =  yTQ_1y  =  1  >  0.    Thus  r  satisfies  Mr  =  0,  rTQr  =  0,  and  -yTr  > 
0  ,  whereby  the  optimization  problem  (P3)  of  Step  3   is  unbounded.   From 
Proposition  3.1,  Pw    is  unbounded.   ■ 

Proposition  6.3.  If    (dN  ,  7iN  )    solve  (6.1),  and  1  +  yTdN  >  0,  then  either  dN  =  0,  or 
dNQdN  >  0. 

Proof:    Because  Q  =  Q  -  yyT , 

dNQdN  =  dNTQdN  -  (yTdN)2  ,  (6.5) 

and  from  (6.2)  we  obtain  dNTQdN  =  -  yTdN  .    Substituting  in  (6.5)  yields 

dNQdN  =  -  yTdN   -  (yTdN)2  .     Since   Q    is  positive  semi-definite, 
dNQdN  >  0.    If   dNQdN  =  0  ,  then  we  must  have  yTdN   =  -1,  or    yTdN  =  0. 
If    yTdN  =  -1,  then  this  contradicts  the  hypothesis  that     1  +  yTdN    >  0.    Thus 
yTdN    =  0,  which  implies  from  (6.1)  that  dN  =0.  ■ 

Proof  of  Lemma  6.1.:    From  Proposition  6.1,  we  have  1  +  yTdN  >  0  .  Suppose 
l+yTdN>0and    dN*0.    Then   d  =  dNVk  /  VdNQdN     '    n  =  :rN  /  (1  +  yTdN) ,  and 
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P  =  '\/dNQdN    /  (2>[k  (1  +  yTdN ) ) ,  are  all  well-defined  (by  Proposition  6.3)  and 
satisfy  the  optimality  conditions     dQ  d  =  k,  M  d  =  0,  -y  =  2  (JQ  d  -  MT  k  ,    (5  >  0, 
for  the  optimization  problem  (P3)  of  Step  3.  Thus     d    is  the  direction  of  Step  3  of 
the  algorithm. 

Next  suppose    1  +  yTdN  >  0     and  that  dN  =  0.    Then  -MT  7CN  =  -y ,  and 
d  =  0,    71  =  kn    satisfy  the  optimality  conditions  for  the  optimization  problem  (P3) 
in  Step  3.  By  Proposition  3.2,  the  current  iterate    x  solves  Pw  . 

Finally,  suppose     1  +  yTdN  =  0.    Then  from  Proposition  6.2,  we  conclude  that 
the  optimization  problem  (P3)  of  Step  3  is  unbounded  and   Pw  is  unbounded.  ■ 

VII.    Inner  and  Outer  Ellipsoids  at  an  approximate  w-center  point    x 

A 

One  of  the  special  features  of  the  w-center    x    of  a  polyhedral  system  X  is  the 

A 

fact  that  there  exist  ellipsoids  EIN  and    EOUT  ,  with  center  at    x  ,  such  that 
EIN  c  "X,  c  EOUT    and     EIN  =(  w/(l  -  w))  EOUT    ,  see    Theorem  2.1  of  Part  I. 

A 

Although  the  iterates  of  the  algorithm  of  Section  II  will  converge  to    x  ,  there  may  not 

A 

be  finite  termination,  and  in  fact  the  solution     x     may  involve  irrational  data.   A 
natural  question  is  whether  one  can  construct  good  ellipsoids    F]N  and    FOUT    about 

A 

points  near    x  ,  with  the  property  that  FIN  c  "X  a  FOUT    ,  and  FOUT  =  c  ■  FIN  , 
where  c  =  0(1  /  w).     The  main  result  of  this  section  answers  this  question  in  the 
affirmative: 
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Theorem  7.1.  (Inner  and  Outer  Ellipsoids  at  an  approximate  w-center  point.)  Let    x 

be  the  current  iterate  of  the  algorithm,  let      s  =  b  -  A  x  ,  and  let  y  be  as  defined  in 
Step  4  of  the  algorithm.    Then  if  y  <  .08567,  the  ellipsoids 

FIN  =  {x  €  Rn  |  Mx  =  g,  (x-   x  )TAT  S_1W  S_1A(x-   x  )  <    wj 


and 


TaT  c-lw  c-l 


FOUT  =  Sxe  Rn  |  Mx  =  g,  (x-   x)]A'  S_1W  S_1A(x-   x)  < 


>f- 


12 


-     W 

w 


5^ 


w 


satisfy      FIN  c  X  c  F, 


OUT 


Remark  7.1.    Note  that  (FOUT  -   x  )  = 


1.75-V/-^- 


+    5  I  (FIN  -  x  ) .  Thus  the 


ratio  of  the  scale  of    FOUT  to  F1N  is  less  than  1.75/w  +  5.    If  w  =  (l/m)e, 
w  =  1/m,  and  this  ratio  is  less  than     1.75m  +  5,  which  is  O(m). 


Remark  7.2.   The  number  of  iterations  of  the  algorithm  needed  to  produce  y  <  .08567 

A 

is  bounded  if  Pw  is  bounded.    Let      x    be  the  optimal  solution  to  Pw  .    Then  if  x° 

is  the  initial  value  of  x  in  the  algorithm,  we  must  have  y  <  .08567  after  at  most 

(1-w]/f(x)-F(x0A 
(w)     \     -0033      I    iterations  .    This  follows  from  Lemma  4.2(i).  If    F*  is  any 

A 

finite  upper  bound  on  the  value  of  F(  x  )  produced  at  Step  4  of  the  algorithm,  then 
y  <  .08567  after  at  most 


(l-  w)/F*-F(xc 


w 


.0033 


iterations. 


The  proof  of  Theorem  7.1   is  a  consequence  of  the  following  sequence  of 
propositions  and  lemmas. 
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Proposition  7.1.    Suppose    x  €  int  X    is  given,  and  let    s  =  b  -  A  x.    Let  x  e  Rn 
satisfy    Mx  =  g    and    (x  -   x  )TAT  S_1W  S^AC  x  -  x)  <    52  w ,  where  6<1. 
Then    x    e  int X,  and  if      s=b-Ax,    (  x-  xjWWS'U  (  x -  x)  S 
52  w/(l-5)2  . 


Proof:  By  supposition,  Mx  =  g.     Let  s  =  b  -  Ax  .  We  first  must  show  that      s  >  0, 
which  will  imply      x    e  intX  .  By  supposition  above  ,     (  s  -   s)  S^W  S"^  s  -  s) 


m 


<    62  w ,  so  that  V  Wj 
i=l 


;i-si> 


V.        i     J 


<  52  w  .    Because    w  <  w4 ,  i  =  1, . . . ,  m,  we 


have 


\J£ 


v< 


u 


<  5  <  1,  so  that  Sj  >  0 ,  i  =  1, . . . ,  m.    This  shows  that      x    e  int  X. 

1 


si  si 

Furthermore,  we  obtain      -=■    >  1-5  ,  so  that   —  < 

si  sj         1-8 


Next,  note  that 


m 


(  x-  x)TATS-1WS-1A(  x-  x)  =     £ 


W; 


rsjY 


i  =  l 


I      S:  ~    S-   V 


,2    m 


~  j  a Wi 


SJ-  Si 


v 


62 


V     "i     J 


(1-6)2 


^siy 


w    . 


1         1 


v.     i     y 
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_  A 

Lemma  7.1.     Let      x  e  int  X     be  given  ,  and  let    x    be  the  w-center  of  X  ,  and 


suppose  that 

(x-   x)TAT  S-!W  S_1A(x-  x  )  <    52  w     for  some    6<1. 
Let    FIN=  {xe  Rn  |  Mx  =  g,  (x  -   x)TAT  S_1W  S_1A(x-  x)  <    wj    and 

FOUT=    xe  Rn  |  Mx  =  g,  (x-   x)TAT  S^W  S_1A(x-   x)  <  (1  +  5)2  A/     ~-W  + W^~ 

I  ^  \       w         1  -  8  ^ 

Then    FIN  c  X  c  FOUT    . 


Proof  of  Lemma  7.1.:    Let    x  be  an  element  of  FIN  .    Then  by  the  same  argument  as 

A 

in  Proposition  7.1,  we  have    x  e   X  ,  so  that      FIN  c  X  .  Because     x     is  the 
w-center  of  X  ,  then 

(x -  x  )TATS'lVJ S-!A(x  -  x  )  <  (1  -   w)/w    for  any    xe   X  ,  (7.1) 

A  A 

where       s  =  b  -  Ax  ,  from  Theorem  2.1  of  Part  I. 

_        A 

Also,  by  Proposition  7.1,  with    x  =  x  ,  we  have 

_     A  A  A  A 

(  x-x)TATS-1WS-1A(  x-x)    <    62w/(l-8)2-  <7-2>- 

Taking  the  square-roots  of  (7.1)  and  (7.2)  and  noting  the  triangle  inequality  for 
norms,  we  have 


^2 


(x-  x)TATS-1WS-1A(x-   x)  < 


-  w  5*V  w 

= —  + 


N1 


w  1-8 


for  every  x  e   X  .     (7.3) 


A  A 

si  si 

Next,  note  that  from  the  hypothesis  of  the  lemma,     -=■  -  1  <  8 ,  and  so    -=■  <  1  +  8 , 


si  s< 


i  =  1  , .  .  . ,  m.    Now  let    x  e   X  be  given,  and  let  s  =  b  -  Ax.  Then 


m 


(x  -   x  )TAT  S-]W  S^ACx  -   x  )  =  £    ws 


rt.*VV 


i=l  V  °\) 


(s,): 


A  A  ( 

(l  +  6)2(x-  x)TATS-1WS-1A(x-  x)      <  (1+5)2 


<- 


-   w         8 

-= —  + 
w 


1-8 


4 


w 


completing  the  proof. 
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Lemma  7.2.    If     x  is  the  current  iterate  of  the  algorithm  and  if  y  <  .08567  at  Step  4, 

a  a      _  —        —       a      _  _ 

and  if     x    is  the  w-center  of  X  ,  then  (x-   x  )TAT  S_1W  S"'A(x-   x  )  <  .55   w. 

A  A  _ 

Proof:  Let    z  =  g(x  )  where    g(x)  is  given  in  (3.3)  of  Part  I.  Let    Q    be  as  given  in 

A       —  t  ~    A       — 
Step  2  of  the  algorithm,  and  let  P2  =  (  z  -   x  )T  Q  (  z  -   x  )  .  Then  substituting  h  =  .5 

in  expression  (4.2) ,  we  obtain  from  Lemmas  4.5  and  4.6   that  if  y  <  .08567,  then 

P  <  hVk    =  .5^Jk  .     Let    d  be  the  direction  defined  in  Step  3  of  the  algorithm  and  let 

a        a        _  _      _ 

d   =  z  -  x  .    Then  because    x  +   d  maximizes  -yTz  over 

ze  E1N  =  (ze  Rn  |  Mz  =  g,  (z-  x)T  Q  (z  -  x)  <  k}    then  -yT(  x  +  ((3/Vk)  d)  > 
-yT(  x±  d  ).    Thus  ±yTd  <  (-p/Vk)yT  d  .  But  y  =  -yT  d/k,  so  ±yTd  <  pyVk  . 
Therefore, 

(7.4) 


(yTd  )2  <  p^k  • 


Next,  note  that  since    Q  =  Q  -  yyT  ,  where  Q  =  AT  S'^W  S"]A  ,  we  have 
a        _  A        _        a       _  a        _         f  a        _T  1  .         „A 

( z  -  x  )TQ  (  z  -  x  )  =  (z  -  x  )TQ  (  z  -  x )  +  L  (z  -  x  )TyJ2  =  p2  +  (yTd  )2  < 


A  _ 


A  _ 


p2  +  P2kr  from  (7.4) .    Also,  (x-  x)=  (z-  x)/(l+yT(z-   x))  from  (3.4)  of 
Part  I,  so  that 


A  _ 


(x  -  x  )TQ  (x  -  x  )  <  (  p2  +  p2k72)/(l  +  yTd  )2  <    (p2  +  p2k^)/(l  -  pVky)2 
^  h2k  +  h2kV         k(h2  +  h2-/2)  2  w(h2  +  h2-/2) 


(1  -  hky): 


(1  -  hy): 


(l-hy): 


<  .55   w. 


3  1 


Proof  of  Theorem  7.1. :    We  first  show  that  FIN  c  X   .     Let     x  e  FIN  and  let 


s  =  b  -  Ax  .  It  suffices  to  show  that   s  >  0.    Because     x  e  FIN  ,  then 

m     fti       T     - 

\  Wj  -=■  -    1        <  w.     Therefore,  because  wi  ^    w  ,     i  =  1, . . . ,  m, 


i=l 


^si 


1-   -==■     <    l,i  =  l,...,m  ,  so  that  s,  >  0,  i  =  l,...,m.    Thus     F1NcX 

S: 


i 


To  show  that  X  c  FOUT  ,  we  apply     Lemma  7.2,  which  shows  that  if 

A 

Y  <  .08567,  then  the  w-center   x  of   X  must  satisfy 

(x  -   x  )TAT  S_1W  S'1A(x  -  x  )  <  52  w  ,  where  8  =  V^55  .  Next,  applying  6  =  V^55 


in  Lemma  7.1,  we  obtain  the  conclusion  that 

n f 

/ 1  -    w  ;  — 

,  thus  showing 


(1+6)2     k/^-=^-+  - — ^f^~      <    1.75A/^-=^ +5"V 


that  X  c  FOUT  .    ■ 

VIII.    Concluding  Remarks 

Alternative  Convergence  Constants.    Lemma  4.3  asserts  that  at  each  iterate  of 
the  algorithm  that  we  obtain  a  constant  improvement  of  at  least  .0033k  or  a  linear 
convergence  to  the  optimal  objective  value,  with  convergence  constant  .32.   The 
constants  .0033  and  .32  are  derived  in  Section  IV  by  using  the  value    h=  .5  in 
Lemmas  4.4,  4.5,  and  4.6.  If  instead  of  choosing    h  =  .5,  one  chooses    h  =  2,  for 
example,  then  by  parallelling  the  methodology  in  Section  4,  one  obtains  Lemma  4.3 
with  a  constant  improvement  of  at  least   .0133k   or  a  linear  convergence  rate  with 
convergence  constant  .64.  The  choice  of  h  =  .5  was  fairly  arbitrary.  Similar  results 
can  be  had  by  choosing  a  different  value  of  h  to  obtain  different  constants  for  the 
threshold  value  of  y  and  the  relative  sizes  of  FOUT  and  FIN  in  Theorem  7.1. 
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Stronger  Convergence.  Vaidya  [  14  ]    has  shown  that  his  Newton  direction 
algorithm  is  linearly  convergent.    Here,  we  have  extended  his  result  and  have  shown 
that  the  algorithm  of  Section  II  (and  Vaidya's  algorithm)  exhibit  superlinear 
convergence.   A  natural  question  for  future  study  is  whether  one  can  show  the 
algorithm  here  to  be  quadratically  convergent. 

The  behavior  of  y.       In  Step  3  of  the  algorithm,  the  search  direction    d  is 

computed.   In  Step  4,  the  parameter  y  ,  as  a  function  of  x  is  computed,  and  we  can 

write  y  =  yi  x)  .  The  value  of  y(  x)  is  then  used  to  derive  upper  bounds,  a  step  length, 

and  a  guaranteed  improvement  in  the  objective  value.    From  Lemma  4.3,  it  is 

_  _  A 

obvious  that  y(  x)  goes  to  zero  as     x  approaches    x  ,  the  optimal  solution  to  Pw. 

Concerning  the  behavior  of  y(  x)  ,  it  is  natural  to  ask  if  the  level  sets  of    yix)  are 
convex,  if  y(x)  decreases  at  each  iteration,  etc.   The  author  has  demonstrated  examples 
where  a  level  set  of  y(x)  is  not  convex,  and  where    y{x)  increases  at  a  particular 
iteration.   Thus  y(x)  is  not  as  well-behaved  as  one  would  hope  for. 

A 

No  Finite  Termination.   As  pointed  out  in  Section  VII,  the  solution    x    to  the 
w-center  problem  can  have  irrational  components  and  so  the  algorithm  will  not 
stop  after  finitely  many  iterations.   Even  if  the  problem  Pw  is  unbounded,  the 

algorithm  may  never  detect  unboundedness,  and  so  may  not  stop  after  finitely 

many  iterations.  This  is  shown  by  the  example  of  Section  4  of  Bayer  and  Lagarias 

[  1  ]  .   In  that  example,  the  iterates  of  a  Newton  method  with  a  line  search  are  traced 

for  the  w-center  problem,  where  w  =  (1/3, 1/3, 1/3) ,  and 

X  €  {x  €  R2  |  Xj  2  -1,  x1  <  1,  x2  >  oj  ,  and  the  starting  point  is    x  =  (1  /3,  2/3).  The 

(Newton)  direction  defined  by  each  iterate  is  never  a  ray  of  X  ,  and  so  the 
algorithm  will  never  stop. 
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